DATA EFFICIENCY

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2023.tacl-1)

Challenge:	Existing Music AVQA methods rely on dense and unoptimized representations, leading to inefficiencies in the isolation of key information, reduction of redundancy, and prioritization of critical samples.
Approach:	They propose a sparse learning framework specifically designed for Music AVQA to address these challenges.
Outcome:	The proposed framework reduces training time by 28.32% while maintaining accuracy while maintaining state-of-the-art performance on the Music AVQA datasets.

Copied to clipboard

Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, Diyi Yang

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes (2024.naacl-short)

Challenge:	Existing methods for enhancing data efficiency in limited labeled data are limited.
Approach:	They propose to use data augmentation methods to increase the efficiency of limited data learning in NLP.
Outcome:	The proposed methods perform well on topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks.

Copied to clipboard

Harmon Bhasin, Timothy Ossowski, Yiqiao Zhong, Junjie Hu

Query2Prod2Vec: Grounded Word Embeddings for eCommerce (2021.naacl-industry)

Challenge:	Multi-task learning (MTL) for generalist models is a promising direction that offers transfer learning potential.
Approach:	They propose to combine multi-task learning (MTL) with in-context learning (ICL) to build models that can generalize to multiple tasks while being robust to out-of-distribution examples.
Outcome:	The proposed training strategies enable models to learn difficult tasks while mixing in prior tasks, denoted as mixed curriculum.

Copied to clipboard

Federico Bianchi, Jacopo Tagliabue, Bingqing Yu

Representation Learning

Challenge:	Query2Prod2Vec is a model that grounds lexical representations for product search in product embeddings.
Approach:	They propose a model that grounds lexical representations for product search in product embeddings.
Outcome:	The proposed model is more accurate than existing methods from the literature . it is also more efficient than existing embedding methods in the context of high-traffic websites.

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation (2025.findings-acl)

Copied to clipboard

Guofeng Cui, Pichao Wang, Yang Liu, Zemian Ke, Zhu Liu, Vimal Bhat

Machine Translation

Challenge:	Large language models (LLMs) have shown great potential in natural language processing tasks, but their application to machine translation remains challenging due to pretraining on predominantly English-centric datasets.
Approach:	They propose a method that combines reward scores with model confidence to improve model selection for fine-tuning.
Outcome:	The proposed method outperforms existing methods in translation accuracy and data efficiency.

Few-Shot Text Generation with Natural Language Instructions (2021.emnlp-main)

Copied to clipboard

Timo Schick, Hinrich Schütze

Information Extraction & Text Mining

Challenge:	Existing approaches to text generation combine task descriptions and examples with supervised learning.
Approach:	They propose a method for text generation that is based on pattern-exploiting training.
Outcome:	The proposed approach improves on several summarization and headline generation datasets.

Efficient Argument Structure Extraction with Transfer Learning and Active Learning (2022.findings-acl)

Copied to clipboard

Xinyu Hua, Lu Wang

Latent-Variable Generative Models for Data-Efficient Text Classification (D19-1)

Challenge:	Identifying and understanding the argumentative discourse structure in text has been a critical task in argument mining.
Approach:	They propose a context-aware Transformer-based argument structure prediction model that outperforms models that rely on features or only encode limited contexts.
Outcome:	The proposed model outperforms models that rely on features or encode limited contexts on five domains and on peer reviews on five different domains.

Copied to clipboard

Xiaoan Ding, Kevin Gimpel

Swift Cross-Dataset Pruning: Enhancing Fine-Tuning Efficiency in Natural Language Understanding (2025.coling-main)

Challenge:	Generative classifiers offer potential advantages over discriminative classifications, including data efficiency and zero-shot learning.
Approach:	They introduce discrete latent variables into generative story to improve classifiers' performance . they empirically characterize performance of their models on six text classification datasets .
Outcome:	The proposed model outperforms discriminative and generative classifiers on six text classification datasets.

Copied to clipboard

Nguyen Binh Nguyen, Yang He

Meta Learning for Natural Language Processing: A Survey (2022.naacl-main)

Challenge:	Current approaches for fine-tuning datasets rely on expensive sample ranking processes . data set pruning aims to select a subset of a dataset for efficient model training .
Approach:	They propose a method that uses TF-IDF embeddings with geometric median to rapidly evaluate sample importance.
Outcome:	The proposed method significantly reduces training and storage costs while maintaining model effectiveness.

Copied to clipboard

Hung-yi Lee, Shang-Wen Li, Thang Vu

Generative Models for Product Attribute Extraction (2023.emnlp-industry)

Challenge:	Meta-learning is an emerging field in machine learning, but there is no systematic survey of these approaches in NLP.
Approach:	They propose to introduce meta-learning and the common approaches and summarize their work and review their work in the NLP community.
Outcome:	The proposed methods improve performance in many NLP tasks but are limited to domains, languages, countries, or styles.

Copied to clipboard

Ansel Blume, Nasser Zalmout, Heng Ji, Xian Li

Tagging

Challenge:	generative models are used for product attribute extraction, a new field in information extraction and e-commerce.
Approach:	They analyze generative models for product attribute extraction and demonstrate their utility . they perform experiments on Amazon and MAVE product attribute datasets .
Outcome:	The proposed model can generate implicit attribute values, which state-of-the-art models are unable to extract.

Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation (2023.eacl-main)

Copied to clipboard

Mehrad Moradshahi, Sina Semnani, Monica Lam

Multilinguality

Challenge:	Existing low-cost approaches to build a high-quality functioning dialogue agent are limited to a few widely-spoken languages.
Approach:	They propose automatic methods that use ToD training data to build a functioning agent in another language . they compare the method to existing methods that only use a small training set .
Outcome:	The proposed method improves the state-of-the-art in Chinese to English transfer using zero-shot data compared to existing full-shot methods . the proposed method achieves 46.7% and 22.0% in task success rate and dialogue success rate, respectively.

Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment (2024.emnlp-main)

Copied to clipboard

Kun Luo, Minghao Qin, Zheng Liu, Shitao Xiao, Jun Zhao, Kang Liu

RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services (2026.acl-industry)

Challenge:	Pre-trained language models have limited generalization capabilities and performance challenges.
Approach:	They evaluate 15 different backbone LLMs and non-LLMs to evaluate their performance . larger models and extensive pre-training consistently enhance in-domain accuracy and data efficiency .
Outcome:	The results show that larger models and extensive pre-training enhance in-domain accuracy and data efficiency.

Copied to clipboard

Fei zhao, Chonggang Lu, Haofu Qian, Fangcheng Shi, Zijie Meng, Jianzhao Huang, Zheyong Xie, Shaosheng Cao

Robustness in NLP

Challenge:	Social networking services (SNS) are critical infrastructure for global interaction . supervised fine-tuning (SFT) can improve in-domain performance, but it often induces a ”seesaw” trade-off with out-of-domain robustness .
Approach:	They propose an SNS-oriented LLM with a progressive, RL-prioritized post-training paradigm for fast and stable adaptation.
Outcome:	The proposed model improves over the previous 7B model by 2.41 on average . it also yields an 8.74 average gain over its Qwen3-4B base .

RV-Syn: Rational and Verifiable Mathematical Reasoning Data Synthesis based on Structured Function Library (2026.findings-eacl)

Copied to clipboard

Jiapeng Wang, Jinhao Jiang, Zhiqiang Zhang, Jun Zhou, Xin Zhao

Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data (2025.findings-naacl)

Challenge:	Existing methods for generating high-quality reasoning data are limited in quality and availability.
Approach:	They propose a method that constructs mathematical operations and generates verifiable graphs that are back-translated into complex problems.
Outcome:	The proposed method achieves a 6.3% performance gain over existing methods on LLaMA-3-8B and outperforms others with only half the training data (50k vs. 100k).

Copied to clipboard

Juanhui Li, Sreyashi Nag, Hui Liu, Xianfeng Tang, Sheikh Muhammad Sarwar, Limeng Cui, Hansu Gu, Suhang Wang, Qi He, Jiliang Tang

ExtEnD: Extractive Entity Disambiguation (2022.acl-long)

Challenge:	Large Language Models (LLMs) have demonstrated superior language understanding abilities in many real-world NLP applications.
Approach:	They propose a learning-based sample selection method that incorporates signals from both teacher and student to enhance model performance.
Outcome:	The proposed method improves model performance across datasets with higher data efficiency.

Copied to clipboard

Edoardo Barba, Luigi Procopio, Roberto Navigli

Information Extraction & Text Mining

Challenge:	Entity disambiguation (ED) is a task in natural language processing that requires a large pre-trained language model to perform.
Approach:	They propose a local formulation for Entity Disambiguation (ED) that frames this task as a text extraction problem and propose two Transformer-based architectures that implement it.
Outcome:	The proposed model outperforms all its competitors in terms of data efficiency and raw performance on 4 out of 4 benchmarks.

Conformal Feedback Alignment: Quantifying Answer-Level Reliability for Robust LLM Alignment (2026.findings-eacl)

Copied to clipboard

Tiejin Chen, Xiaoou Liu, Vishnu Nandam, Kuan-Ru Liou, Hua Wei

KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference (2021.acl-long)

Challenge:	Existing uncertainty-aware approaches weight preferences, but ignore reliability of the answers being compared.
Approach:	They propose a framework that grounds preference weighting in Conformal Predictions to address this problem.
Outcome:	The proposed framework improves alignment robustness and data efficiency across different datasets.

Copied to clipboard

Qianglong Chen, Feng Ji, Xiangji Zeng, Feng-Lin Li, Ji Zhang, Haiqing Chen, Yin Zhang

Explainability & Interpretability in NLP

Challenge:	Existing approaches in NLP focus on “WHY A” rather than contrastive “WHA NOT B” Experimental results show that contrastive explanations are beneficial to fit the scenarios by clarifying the difference between the predicted answer and other possible wrong ones.
Approach:	They propose to generate contrastive explanations with counterfactual examples in NLI by identifying key phrases from input sentences and using them as key perturbations to generate counterfacts.
Outcome:	The proposed framework improves on SNLI and ETPA models by 91.9%.

GigaCheck: Detecting LLM-generated Content via Object-Centric Span Localization (2026.findings-acl)

Copied to clipboard

Irina Tolstykh, Aleksandra Tsybina, Sergey Yakubson, Aleksandr Gordeev, Vladimir Dokholyan, Maksim Kuprashevich

Exploring Automatically Perturbed Natural Language Explanations in Relation Extraction (2023.findings-acl)

Challenge:	GigaCheck is a framework for AI-generated text detection.
Approach:	They propose a dual-strategy framework for AI-generated text detection . they leverage representation learning of fine-tuned LLMs to discern authorship .
Outcome:	The proposed framework can detect LLM-generated content with high accuracy and accuracy . it can be used in mixed-authorship scenarios and in academic collaborations .

Copied to clipboard

Wanyun Cui, Xingran Chen

SEMIROUTER: Sparse-Data Enhanced Routing for Adaptive Multi-LLM System (2026.eacl-long)

Challenge:	ExpBERT is a new approach to guide neural networks with explanations . previous studies have shown that explanations provide valuable inductive biases that guide models, improving generalization ability and data efficiency.
Approach:	They propose to use a new inductive bias-based model to guide models with explanations.
Outcome:	The proposed explanations achieve comparable results to annotated explanations, but with a significant increase in computational efficiency, 20-30 times faster.

Copied to clipboard

Zijie Wang, Xinyu Yan, Che Wang, Zeng Zihao, Lei Xiao, Wei Yang Bryan Lim

Green & Sustainable NLP

Challenge:	Existing routing methods suffer from poor scalability and dependence on datasets for training . energy footprint is also considered in the decision to implement our new LLM routing framework .
Approach:	They propose a new LLM routing framework that dynamically allocates queries to the most appropriate LLM.
Outcome:	The proposed method improves data efficiency, adaptability, and routing accuracy compared to existing methods.

Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoning (2025.findings-emnlp)

Copied to clipboard

Zhaohui Yang, Chenghua He, Xiaowen Shi, Shihong Deng, Linjing Li, Qiyue Yin, Daxin Jiang

On the Limitations of Simulating Active Learning (2023.findings-acl)

Challenge:	Existing methods for training effective PRMs focus on the first incorrect step and all preceding steps, assuming that all subsequent steps are incorrect.
Approach:	They propose a data annotation method specifically designed to score the long CoT reasoning process by using an LLM-based judger for annotation.
Outcome:	The proposed method improves PRMs' ability to identify effective self-correction behaviors and reasoning based on erroneous steps.

Copied to clipboard

Katerina Margatina, Nikolaos Aletras

From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models (2026.acl-long)

Challenge:	Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation.
Approach:	They propose to simulate active learning by using an already labeled dataset as the pool of unlabeled data.
Outcome:	The proposed model-in-the-loop paradigm can be used to perform experiments with human annotations on-the fly.

Copied to clipboard

Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo

KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Media (2022.naacl-main)

Challenge:	Recent research in mechanistic interpretability has revealed that Large Language models contain disentangled, human-understandable components.
Approach:	They propose a framework that first identifies causal task features through frequency recall and interventional filtering, then selects “Feature-Resonant Data” that maximally activates task features for fine-tuning.
Outcome:	The proposed framework outperforms existing models on mathematical reasoning, summarization, and translation tasks while using only 50% of the data.

Copied to clipboard

Wenqian Zhang, Shangbin Feng, Zilong Chen, Zhenyu Lei, Jundong Li, Minnan Luo

Document Classification for COVID-19 Literature (2020.findings-emnlp)

Challenge:	Existing approaches focus on leveraging textual content to identify stances, while they fail to reason with background knowledge or leverage the rich semantic and syntactic textual labels in news articles.
Approach:	They propose a political perspective detection approach that leverages news text to enable multi-hop knowledge reasoning and incorporates textual cues as paragraph-level labels.
Outcome:	The proposed approach outperforms state-of-the-art methods on two benchmark datasets.

Copied to clipboard

Bernal Jimenez Gutierrez, Jucheng Zeng, Dongdong Zhang, Ping Zhang, Yu Su

Graph-Based Decoding for Task Oriented Semantic Parsing (2021.findings-emnlp)

Challenge:	a global pandemic has made it more important than ever to quickly and accurately retrieve relevant scientific literature for effective consumption by researchers in a wide variety of fields.
Approach:	They analyze a LitCovid dataset to find out how classification models can help organize COVID-19 research papers.
Outcome:	The proposed model outperforms all baseline models on the LitCovid dataset . it also outperformed BioBERT and other models with micro-F1 and accuracy scores of 86% and 75% .

Copied to clipboard

Jeremy Cole, Nanjiang Jiang, Panupong Pasupat, Luheng He, Peter Shaw

Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction (2025.findings-acl)

Challenge:	Existing paradigms for semantic parsing are sequence-to-sequence and AMR parsers.
Approach:	They propose to formulate parsing as a sequence-to-sequence task using graph-based decoding techniques developed for syntactic parsers.
Outcome:	The proposed approach is competitive with sequence decoders on the standard setting and offers significant improvements in data efficiency and data availability.

Copied to clipboard

Yuxin Jiang, Yufei Wang, Chuhan Wu, Xinyi Dai, Yan Xu, Weinan Gan, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, Wei Wang

Low-Resource NLP

Challenge:	Existing methods for generating and curating high-quality instruction-tuning data rely heavily on the quality of seed data or strong assumptions about the structure and content of web documents.
Approach:	They propose a fully automated framework for synthesizing high-quality instruction-tuning (IT) data directly from raw web documents with minimal assumptions.
Outcome:	The proposed framework outperforms state-of-the-art baselines by 16.65% across four instruction-following benchmarks.

LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issues (2026.acl-long)

Copied to clipboard

Fanyu Wang, Xiaoxi Kang, Paul Burgess, Aashish Srivastava, Chetan Arora, Adnan Trakic, Lay-Ki Soon, Md Khalid Hossain, Lizhen Qu

ALICE: Active Learning with Contrastive Natural Language Explanations (2020.emnlp-main)

Challenge:	Large language models (LLMs) have impressive reasoning capabilities, but their precision remains inadequate.
Approach:	They propose a framework that integrates neural generation with statistical reasoning to improve the accuracy of large language models.
Outcome:	The proposed framework achieves interpretability through transparent feature weighting while maintaining data efficiency through correlation-based statistical classification.

Copied to clipboard

Weixin Liang, James Zou, Zhou Yu

Interactive Symbol Grounding with Complex Referential Expressions (2022.naacl-main)

Challenge:	Annotating a large dataset with annotations is costly and infeasible.
Approach:	They propose an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning.
Outcome:	The proposed framework outperforms baseline models trained with 40-100% more training data on bird species classification and social relationship classification tasks.

Copied to clipboard

Rimvydas Rubavicius, Alex Lascarides

Visual Data in NLP

Challenge:	Existing work on symbol grounding models (grounders) uses lazy few-shot learning to relate open-class words like green and above to their visual percepts; and symbolic reasoning with closed-class word categories like quantifiers and negation.
Approach:	They propose a procedure for learning to ground symbols from a sequence of stimuli consisting of an arbitrarily complex noun phrase and its designation in the visual scene.
Outcome:	The proposed procedure is based on a visual reference resolution task in which the learner is unaware of concepts that are part of the domain model and how they relate to visual percepts.

FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking (2024.findings-emnlp)

Copied to clipboard

Zhuoer Wang, Leonardo Ribeiro, Alexandros Papangelis, Rohan Mukherjee, Tzu-Yen Wang, Xinyan Zhao, Arijit Biswas, James Caverlee, Angeliki Metallinou

MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)

Challenge:	Existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user’s request.
Approach:	They propose an output-side optimization approach that uses state-tracked constrained decoding and a lightweight model as the discriminator to rerank the beam-searched candidate generations of the large language model.
Outcome:	The proposed approach outperforms existing supervised and in-context learning approaches in the DSTC8 and API Bank datasets in terms of call generation accuracy, inference efficiency, and context efficiency.

Copied to clipboard

Qizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li, Honglin Lin, Chenlin Ming, Xin Gao, Conghui He, Rui Yan

Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills (2022.acl-long)

Challenge:	Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications .
Approach:	They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis.
Outcome:	The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency.

Copied to clipboard

Ori Yoran, Alon Talmor, Jonathan Berant

Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (2024.findings-acl)

Challenge:	Large pre-trained language models struggle in tasks that require reasoning . recent work shows that they struggle in performing symbolic reasoning operations without substantial amounts of additional data.
Approach:	They propose to leverage semi-structured tables and generate at scale question-paragraph pairs where answering the question requires reasoning over multiple facts in the paragraph.
Outcome:	The proposed model outperforms T5, a popular pre-trained encoder-decoder model, on reasoning-focused reading comprehension datasets.

Copied to clipboard

Lvxue Li, Jiaqi Chen, Xinyu Lu, Yaojie Lu, Hongyu Lin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun

Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task (2022.findings-emnlp)

Challenge:	In-context learning (ICL) has gained considerable attention due to its data efficiency and task adaptability.
Approach:	They propose to de-biase demonstration bias in in-context learning by focusing on semantic ambiguity induced by demonstrations and reducing the semantic hazard.
Outcome:	The proposed methods significantly improve performance on six datasets.

Copied to clipboard

Shailaja Keyur Sampat, Pratyay Banerjee, Yezhou Yang, Chitta Baral

NSL-MT: Linguistically Informed Negative Samples for Efficient Machine Translation in African Low-Resource Languages (2026.findings-acl)

Challenge:	‘Actions’ play a vital role in how humans interact with the world and therefore require the capability to perform ‘Reasoning about Actions & Change’ (RAC).
Approach:	They propose an encoder-decoder architecture to learn the representation of actions as vectors and combine it with existing modality parsers and a scene graph question answering model to evaluate the proposed system.
Outcome:	The proposed system performs better than baselines on the CLEVR_HYP dataset and compares with existing parsers and a scene graph question answering model.

Copied to clipboard

Mamadou K. Keita, Christopher M Homan, Huy Le

Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration (2025.acl-long)

Challenge:	In low-resource settings, models encounter too few examples to reliably distinguish grammatical patterns from noise.
Approach:	They propose a negative space learning machine translation (NSL-MT) method that augments limited parallel data with synthetically generated violations of the target language’s grammar and explicitly penalizes the model when it assigns high probability to these violations.
Outcome:	The proposed method delivers 3-12% BLEU gains for well-performing models and 56-89% gains for models lacking decent initial support.

Copied to clipboard

Tianyi Bai, Ling Yang, Zhen Hao Wong, Fupeng Sun, Xinlin Zhuang, Jiahui Peng, Chi Zhang, Lijun Wu, Qiu Jiantao, Wentao Zhang, Binhang Yuan, Conghui He

Unlocking the Potential of Model Merging for Low-Resource Languages (2024.findings-emnlp)

Challenge:	Efficient data selection is crucial to accelerate the pretraining of language models . limited research has addressed the inherent conflicts between data selection methods .
Approach:	They propose a multi-actor collaborative data selection mechanism that prioritizes data based on its specific criterion and updates prioritization rules using the current state of the model.
Outcome:	The proposed model accelerates convergence in LM pretraining and achieves an average relative performance gain of 10.5% across multiple language model benchmarks.

Copied to clipboard

Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songfang Huang, Dongyan Zhao, Yansong Feng

Levenshtein Training for Word-level Quality Estimation (2021.emnlp-main)

Challenge:	Adapting large language models (LLMs) to new languages requires continual pre-training followed by supervised fine-tuning.
Approach:	They propose a model merging solution that integrates LLMs with distinct capabilities into a single model without additional training.
Outcome:	The proposed model merging outperforms CT-then-SFT in low-resource languages with scarce data.

Copied to clipboard

Shuoyang Ding, Marcin Junczys-Dowmunt, Matt Post, Philipp Koehn

The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025.findings-acl)

Challenge:	a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision.
Approach:	They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation.
Outcome:	The proposed method performs better under data-constrained and unconstrained conditions than existing methods.

Copied to clipboard

Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin

Self-Generated Critiques Boost Reward Modeling for Language Models (2025.naacl-long)

Challenge:	a recent study shows that process reward models can make mistakes, leading to wrong conclusions.
Approach:	They propose a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge to improve model performance and data efficiency.
Outcome:	The proposed model outperforms existing open-source alternatives and provides practical guidelines for future research.

Copied to clipboard

Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, Melanie Kambadur, Dhruv Mahajan, Rui Hou

CARER - ClinicAl Reasoning-Enhanced Representation for Temporal Health Risk Prediction (2024.emnlp-main)

Challenge:	Existing reward models produce scalar scores and struggle to incorporate critiques in a natural language format.
Approach:	They propose a framework that predicts critiques and rewards using self-generated critiques without extra supervision.
Outcome:	The proposed framework improves reward modeling accuracy by 3.7%-7.3% compared to standard reward models and LLM judges.

Copied to clipboard

Tuan Nguyen, Thanh Huynh, Minh Hieu Phan, Quoc Viet Hung Nguyen, Phi Le Nguyen

Representation Learning

Challenge:	Existing deep learning methods require large datasets to achieve high generalizability.
Approach:	They propose a framework that enhances deep learning models with clinical rationales derived from medically proficient Large Language Models.
Outcome:	The proposed framework outperforms state-of-the-art models on two tasks using two popular EHR datasets by up to 11.2%.

Constraint-based Learning of Phonological Processes (D19-1)

Copied to clipboard

Shraddha Barke, Rose Kunkel, Nadia Polikarpova, Eric Meinhardt, Eric Bakovic, Leon Bergen

Syntactic Text Processing

Challenge:	Phonological processes govern the way speech sounds in natural languages change depending on context . a novel approach to learning phonological processes from related utterances is proposed .
Approach:	They propose an unsupervised approach to learning phonological processes from related utterances . they encode the problem into Boolean constraints that enable data efficiency and fast inference .
Outcome:	The proposed approach achieves high accuracy at interactive speeds on phonology problems and datasets.

Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models (2024.findings-emnlp)

Copied to clipboard

Yiming Chen, Xianghu Yue, Xiaoxue Gao, Chen Zhang, Luis Fernando D’Haro, Robby Tan, Haizhou Li

Agentic Episodic Control (2026.findings-acl)

Challenge:	Existing evaluations of audio large language models focus on single audio inputs, but real-world applications often require processing multiple audio streams simultaneously.
Approach:	They propose a multi-audio evaluation benchmark that combines 20 audio inputs from 11 audio tasks to capture audio context.
Outcome:	The proposed model outperforms baseline models and achieves high data efficiency without human annotations.

Copied to clipboard

Xidong Yang, Wenhao Li, Junjie Sheng, Yun Hua, Haosheng Chen, Chuyun Shen, Xiangfeng Wang

Green & Sustainable NLP

Challenge:	Existing methods for reinforcement learning (RL) are limited by poor data efficiency and weak generalization.
Approach:	They propose a novel architecture that integrates large language models into episodic RL.
Outcome:	The proposed architecture achieves 2–6 higher data efficiency than baselines and is the only method to solve complex tasks like UnlockLocal with over 90% success.

Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference (2020.emnlp-main)

Copied to clipboard

Xiaoan Ding, Tianyu Liu, Baobao Chang, Zhifang Sui, Kevin Gimpel

Text Classification

Challenge:	Recent work has shown advantages of generative classifiers in terms of data efficiency and robustness.
Approach:	They propose a generative classifier for natural language inference (NLI) they compare it to discriminative models and large-scale pretrained models like BERT .
Outcome:	The proposed classifier outperforms discriminative and pretrained baselines across several challenging NLI experimental settings, including small training sets, imbalanced label distributions, and label noise.

Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision (2025.findings-emnlp)

Copied to clipboard

Tej Deep Pala, Panshul Sharma, Amir Zadeh, Chuan Li, Soujanya Poria

Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization (2021.emnlp-main)

Challenge:	Large Language Models (LLMs) are prone to hallucination, especially during multihop tasks.
Approach:	They propose a hierarchical, erroraware discriminative PRM that classifies math errors at each step and combines finegrained signals to estimate step correctness.
Outcome:	The proposed model outperforms the prior best in a new stateof-theart PRMScore of 67.7 on a 400Ksample dataset .

Copied to clipboard

Inbar Oren, Jonathan Herzig, Jonathan Berant

FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Tasks (2025.findings-emnlp)

Challenge:	Recent research shows that automatic generation of synthetic utterance-program pairs can alleviate the first problem, but its potential for the second has thus far been under-explored.
Approach:	They propose to generate synthetic utterance-program pairs for improving compositional generalization in semantic parsing by using structurally-diverse examples.
Outcome:	The proposed approach leads to dramatic improvements in compositional generalization and moderate improvements in the traditional i.i.d setup.

Copied to clipboard

Sabrina McCallum, Amit Parekh, Alessandro Suglia

Annotator-Centric Active Learning for Subjective NLP Tasks (2024.emnlp-main)

Challenge:	Current approaches to embodied AI tend to learn policies from expert demonstrations, but without a mechanism to evaluate the quality of demonstrated actions, they are limited to learning from optimal behaviour or risk replicating errors and inefficiencies.
Approach:	They propose to embed language feedback into a Transformer-based policy and optionally complement the traditional next action prediction objective with auxiliary self-supervised learning objectives for feedback prediction.
Outcome:	The proposed method improves agents’ compositional generalisation abilities and robustness on a range of embodied Vision-and-Language tasks in a custom babyAI-XGen environment.

Copied to clipboard

Michiel Van Der Meer, Neele Falk, Pradeep Murukannaiah, Enrico Liscio

Self-Training Large Language and Vision Assistant for Medical Question Answering (2024.emnlp-main)

Challenge:	Annotator-centric active learning addresses the high costs of collecting human annotations by strategically annotating the most informative samples.
Approach:	They propose annotator-centric active learning which incorporates an annotation strategy following data sampling to approximate the full diversity of human judgments.
Outcome:	The proposed approach improves data efficiency and performs well in annotator-centric evaluations.

Copied to clipboard

Guohao Sun, Can Qin, Huazhu Fu, Linwei Wang, Zhiqiang Tao

Visual Data in NLP

Challenge:	Existing methods for collecting medical data are expensive and time-consuming.
Approach:	They propose a method to train a large-scale LVLM capable of auto-generating medical visual instruction data to improve data efficiency.
Outcome:	The proposed method shows that it performs well across three major visual question answering (VQA) benchmarks.

Can Language Models Induce Grammatical Knowledge from Indirect Evidence? (2024.emnlp-main)

Copied to clipboard

Miyu Oba, Yohei Oseki, Akiyo Fukatsu, Akari Haga, Hiroki Ouchi, Taro Watanabe, Saku Sugawara

Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models (2025.findings-emnlp)

Challenge:	Recent advances in language models have shown remarkable progress in various tasks.
Approach:	They introduce a dataset that incorporates wug words and inject them into pretraining data and evaluate them on evaluation data.
Outcome:	The proposed model does not induce grammatical knowledge even after repeated exposure to instances with the same structure but differing only in lexical items from evaluation instances in certain language phenomena.

Copied to clipboard

Makesh Narsimhan Sreedhar, Traian Rebedea, Christopher Parisien

DORM: Preference Data Weights Optimization for Reward Modeling in LLM Alignment (2025.findings-emnlp)

Challenge:	Recent research shows that reasoning-based language models offer significant benefits for LLM safety and guardrail applications.
Approach:	They conduct an analysis of reasoning-based guardrail models for content moderation . they find reasoning models exhibit strong sample efficiency and inference efficiency .
Outcome:	The reasoning-based guardrail models show strong performance across domains . the models achieve competitive performance with significantly fewer training examples .

Copied to clipboard

Rongzhi Zhang, Chenwei Zhang, Xinyang Zhang, Liang Qiu, Haoming Jiang, Yuchen Zhuang, Qingru Zhang, Hyokun Yun, Xian Li, Bing Yin, Tuo Zhao, Chao Zhang

Challenge:	Existing approaches to align large language models with human preferences are noisy and varying in importance of preference samples.
Approach:	a new method enhances reward modeling by learning to dynamically weigh preference data.
Outcome:	a new method improves the performance of large language models with human preferences . it initializes data importance and iteratively refines them to maximize validation performance.

SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation (2026.acl-long)

Copied to clipboard

Mahi Luthra, Jiayi Shen, Maxime Poli, Angelo Ortiz Tandazo, Yosuke Higuchi, Youssef Benchekroun, Martin Gleize, Charles-Éric Saint-James, Dongyan Lin, Phillip Rust, Angel Villar-Corrales, null Surya, Vanessa Stark, Rashel Moritz, Juan Pino, Yann LeCun, Emmanuel Dupoux

LaCo: Layer-wise Compensation for Pruned Large Language Models (2026.acl-long)

Challenge:	Empirically, SpidR-Adapt achieves rapid gains in phonemic discriminability and downstream spoken language modeling scores . current self-supervised learning models require thousands of hours of training data to learn meaningful linguistic representations.
Approach:	They propose a bi-level optimization framework for rapid adaptation of speech units to new languages using minimal unlabeled data.
Outcome:	The proposed model achieves rapid gains in phonemic discriminability and spoken language modeling scores . it surpasses in-domain toplines after training on less than 1h of target-language audio .

Copied to clipboard

Yingen Liu, Fan Wu, null Panxuyan, Ruihui Li, Zhuo Tang, Kenli Li

Dynamic Jointly Batch Selection for Data Efficient Machine Translation Fine-Tuning (2025.emnlp-main)

Challenge:	Existing methods for predicting performance degradations of Large Language Models (LLMs) neglect the structural distortions caused by sparsity.
Approach:	They propose a framework that reorients the recovery paradigm from global adaptation to hierarchical representation alignment by sequentially optimizing each layer to reconstruct the model's hidden states.
Outcome:	The proposed framework surpasses parameter-efficient baselines in perplexity reduction and zero-shot reasoning.

Copied to clipboard

Mohammad Amin Ghanizadeh, Mohammad Javad Dousti

Studying the Role of Input-Neighbor Overlap in Retrieval-Augmented Language Models Training Efficiency (2025.emnlp-main)

Challenge:	Data quality and effective selection are key to improving machine translation performance . study focuses on fine-tuning models using a batch selection strategy .
Approach:	They propose a data selection methodology for fine-tuning machine translation systems that leverages the synergy between a learner model and a pre-trained reference model to enhance overall training effectiveness.
Outcome:	The proposed method improves training efficiency by up to fivefold compared to baseline methods.

Copied to clipboard

Ehsan Doostmohammadi, Marco Kuhlmann

Improving Dialogue State Tracking through Combinatorial Search for In-Context Examples (2025.acl-long)

Challenge:	Existing studies have shown that retrieval-augmented language models can perform better than larger models, but the optimal degree of overlap remains unclear.
Approach:	They investigate how varying levels of query–context overlap affect model performance . they specifically generate synthetic context through paraphrasing queries .
Outcome:	The proposed model can improve test-time perplexity and training time by 40% without compromising performance.

Copied to clipboard

Haesung Pyun, Yoonah Park, Yohan Jo

Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning (2025.emnlp-main)

Challenge:	Existing methods for training dialogue state tracking data are suboptimal . existing methods rely on suboptimized data, resulting in poor performance .
Approach:	They propose a method that scores effective in-context examples based on their combinatorial impact on DST performance.
Outcome:	The proposed method achieves a 20% gain in data efficiency and generalizing well to the SGD dataset.

Copied to clipboard

Ziyang Wang, Jaehong Yoon, Shoubin Yu, Md Mohaiminul Islam, Gedas Bertasius, Mohit Bansal

Visual Data in NLP

Challenge:	Despite advances in reinforcement learning, data collection and fine-tuning remain costly and hard to scale.
Approach:	They propose a video-adaptive test-time scaling strategy that combines RL with a supervised fine-tuning strategy to improve video reasoning capability.
Outcome:	The proposed method surpasses existing models by 2.4% in accuracy using only 3.6% training samples.

Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation (2026.acl-long)

Copied to clipboard

Wei-Cheng Tseng, Xuanru Zhou, Mingyue Huo, Yiwen Shao, Hao Zhang, Dong Yu

From Selection to Refinement: Iterative Optimization for Instruction Data (2026.acl-long)

Challenge:	Existing methods for learning general-purpose audio representations are limited in scope and coverage of audio attributes.
Approach:	They propose to use a 10.7M caption dataset to compare ALP with captioning . they find that contrastive learning yields competitive, transferable representations .
Outcome:	The proposed model yields competitive, transferable representations, while captioning exhibits better scalability.

Copied to clipboard

Hang Hu, Ziyan Liu, Rujie Wen, Ruihui Hou, Xueyan Wu, Mu Zhang, Jianxing Yu, Tong Ruan, Jingping Liu